Skip to content

test(prose): a case starts where a session starts#563

Open
leeovery wants to merge 1 commit into
feat/prose-tests-deterministic-checksfrom
test/prose-cases-start-where-sessions-start
Open

test(prose): a case starts where a session starts#563
leeovery wants to merge 1 commit into
feat/prose-tests-deterministic-checksfrom
test/prose-cases-start-where-sessions-start

Conversation

@leeovery

@leeovery leeovery commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • No session begins at a navigation skill, a processing skill, or a reference. workflow-start is the user's way in; after a context clear the bridge's plan file invokes a workflow-*-entry skill, or workflow-discovery for an epic continuation. Verified against all six bridge continuation references.
  • Four cases ignored thatcontinue-feature-routes-to-discussion, continue-bugfix-routes-to-investigation (navigation skills), discussion-session-substituted (processing skill), root-cause-validation-clean-verdict (a reference). A walk carries only the context it accumulates, so a mid-session start leaves the world right and the reading that got there missing. The prose is judged under conditions it was never written for, and the verdict is worthless either way.
  • continue-feature had the flaw written into its own expected path: "initialisation is casing only — no boot and no migrations, which the entry skill has already guaranteed." A case documenting the context it doesn't have. It's also the case that consumed a day producing findings that dissolved on inspection.
  • Retired, not repaired. Stubs stay — they're a reusable library, not case scenery.
  • entry is now required and validated in lib/cases.cjs, so it can't drift back in on authoring discipline.
  • Recorded as P9/P10 in design/prose-tests.md, with the same reasoning applied to where a walk ends: stop at a gate and the delta is empty, claims decay into descriptions of a display, and the asserter compares the walker's account of what it showed against the case's account of what it should have shown — no independent ground truth anywhere.

Coverage note

Coverage of continue-* routing goes with these cases and is not replaced here. It belongs in a case that starts at workflow-start and walks through selection until the route lands real state — the "walk further" shape. Flagging rather than silently dropping.

Test plan

  • Corpus 9/9 valid under the new rule
  • node --test prose suites — 64/64, including 9 new tests covering each accepted and rejected entry class, a plausible-but-absent skill, and the live corpus
  • Conventions lint clean
  • CLAUDE.md and design/prose-tests.md updated

🤖 Generated with Claude Code

Stack

  1. docs(design): prose-tests programme design log #544
  2. feat(prose-tests): the framework — cases, worlds, runner, skill #545
  3. test(prose): feature happy-path corpus — five worlds, seven cases #546
  4. test(prose): bugfix corpus — the investigation-centric surfaces #548
  5. test: retry recursive teardown removals — kill a class of phantom failures #549
  6. fix(entry-skills): close the handoff fences — six files render their arms wrong #550
  7. docs: a contributing page for working on the system #551
  8. fix(entry-skills): every handoff arm says to invoke the skill #552
  9. fix(implementation): environment setup belongs to the setup reference alone #553
  10. fix(prose-tests): the asserter is told which substitutions were armed #554
  11. feat(prose-tests): the mid-flow substitution, and a world only prose can describe #555
  12. test(prose): claims assert consequences, not what was displayed #556
  13. feat(prose-tests): record everything the agents do, results included #557
  14. fix(discussion-entry): the handoff reports the source it actually had #558
  15. fix(prose-tests): the stop hook records, and names the model that walked #559
  16. fix(prose-tests): command output was never actually recorded #560
  17. feat(prose-tests): judge the walk as told, not the summary returned #561
  18. feat(prose-tests): decide in code what an agent should not be deciding #562
  19. test(prose): a case starts where a session starts #563 👈 current
  20. feat(prose-tests): walk on Sonnet, judge on Opus, escalate a failure #564
  21. test(prose): give the eight read-only cases something that can fail #565
  22. test(prose): only walks that can be observed, and checks that survive the trip #566
  23. fix(prose-tests): the verdict names only the model the record names #567
  24. test(prose): discovery, walked to the point where work first exists #568
  25. fix(prose-tests): the asserter judges which of prose or walker was at fault #569
  26. docs(conventions): a step whose reference routes every exit still signposts #570
  27. test(prose): discovery's epic arm, to the same durability boundary #571

No session ever begins at a navigation skill, a processing skill, or a
reference. `workflow-start` is the user's way in; after a context clear
the bridge's plan file invokes a `workflow-*-entry` skill, or
`workflow-discovery` for an epic continuation. That is the whole list.

Four cases ignored it — two opened at `workflow-continue-*`, one at
`workflow-discussion-process`, one inside `root-cause-validation.md`. A
walk carries only the context it accumulates, so starting one in the
middle leaves the world right and the reading that got there missing.
The prose is then judged under conditions it was never written for, and
the verdict is worthless whichever way it lands.

continue-feature had the flaw written into its own expected path:
"initialisation is casing only — no boot and no migrations, which the
entry skill has already guaranteed". A case documenting the context it
does not have. It is also the case that consumed a day of investigation
producing findings that dissolved on inspection.

The four are retired rather than repaired. Their stubs stay — stubs are a
reusable library, not case-scenery.

`entry` is now required and validated, so this cannot drift back in by
authoring discipline alone. The same reasoning governs the far end of a
walk, recorded as P9: stop at a gate and the delta is empty, the claims
decay into descriptions of a display, and the asserter is left comparing
the walker's account of what it showed against the case's account of
what it should have shown — with no independent ground truth anywhere.

Coverage of continue-* routing goes with them, and is not replaced here:
it belongs in a case that starts at workflow-start and walks through
selection until the route lands real state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leeovery
leeovery force-pushed the test/prose-cases-start-where-sessions-start branch from 52d39d6 to 5e3be38 Compare July 26, 2026 18:52
This was referenced Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant